Skip to content

Comments

Add Yappi Profiling#38

Open
daharoni wants to merge 7 commits intomainfrom
add-profiler-and-vis
Open

Add Yappi Profiling#38
daharoni wants to merge 7 commits intomainfrom
add-profiler-and-vis

Conversation

@daharoni
Copy link
Member

Follows #37

Summary

Function-level profiling with yappi and SnakeViz visualization to complement existing line_profiler for pipeline performance analysis.

Changes

  • Add profiling utilities (src/indeca/utils/profiling.py)

    • Minimal yappi_profile context manager for pipeline-level profiling
    • Supports wall-clock and CPU time measurements
    • Outputs pstat format compatible with snakeviz
  • Add benchmark scripts (benchmarks/profile/)

    • profile_pipeline_small.py - 10 cells × 1K frames (quick iterations)
    • profile_pipeline_medium.py - 50 cells × 5K frames (realistic workload)
    • profile_pipeline_large.py - 100 cells × 10K frames (comprehensive)
  • Add dependencies

    • yappi>=1.6.0 (main dependencies)
    • snakeviz>=2.2.0 (dev dependencies)
  • Update .gitignore

    • Ignore *.prof files

Usage

from indeca.utils.profiling import yappi_profile

with yappi_profile("pipeline.prof"):
    pipeline_bin_new(Y, config=config, spawn_dashboard=False)

# Visualize: snakeviz pipeline.prof
# Run benchmark
python benchmarks/profile/profile_pipeline_small.py --profile

# Visualize
snakeviz benchmarks/profile/output/profile_pipeline_small.prof

Testing

  • Verified yappi profiling generates valid pstat files
  • Benchmark scripts run successfully with profiling enabled
  • snakeviz can visualize generated profiles

@daharoni daharoni marked this pull request as ready for review December 14, 2025 02:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant